Prediction of Tactile Perception from Vision on Deformable Objects

Workshop on Robotic Manipulation of Deformable Objects (ROMADO) in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020)Through the use of tactile perception, a manipulator can estimate the stability of its grip, among others. However, tactile sensors are only activated upon contact. In contrast, humans can estimate the feeling of touching an object from its visual appearance. Providing robots with this ability to generate tactile perception from vision is desirable to achieve autonomy. To accomplish this, we propose using a Generative Adversarial Network. Our system learns to generate tactile responses using as stimulus a visual representation of the object and target grasping data. Since collecting labeled samples of robotic tactile responses consumes hardware resources and time, we apply semi-supervised techniques. For this work, we collected 4000 samples with 4 deformable items and experiment with 4 tactile modalities.This work was supported in part by the Spanish Government and the FEDER Funds [BES-2016-078290] and in part by the European Commission [COMMANDIA SOE2/P1/F0638], action supported by Interreg-V Sudoe

Reutilización de datos abiertos de investigación para el desarrollo de sistemas de recomendación

Los datos abiertos se están convirtiendo en una fuente de datos muy útil en diversos ámbitos como el de la movilidad, el turístico o el ámbito periodístico. Sin embargo, se ha trabajado muy poco en uno de los ámbitos más relevantes par a la humanidad: el ámbito de las ciencias de la salud. Esto es debido, sobre todo, a las reticencias que plantea la apertura de datos clínicos y su aparente contradicción con la privacidad de las personas. En este trabajo se plantea como objetivo la creación de un sistema recomendador en salud utilizando datos abiertos con el fin de mostrar la gran oportunidad que representa la reutilización de estos datos abiertos. Para ello se estudiarán los datos abiertos, actualmente disponibles, en salud a nivel nacional e internacional, se analizarán las bases de datos encontradas y, se realizará la extracción de descriptores y caracterización de la información

Generation of Tactile Data from 3D Vision and Target Robotic Grasps

Tactile perception is a rich source of information for robotic grasping: it allows a robot to identify a grasped object and assess the stability of a grasp, among other things. However, the tactile sensor must come into contact with the target object in order to produce readings. As a result, tactile data can only be attained if a real contact is made. We propose to overcome this restriction by employing a method that models the behaviour of a tactile sensor using 3D vision and grasp information as a stimulus. Our system regresses the quantified tactile response that would be experienced if this grasp were performed on the object. We experiment with 16 items and 4 tactile data modalities to show that our proposal learns this task with low error.This work was supported in part by the Spanish Government and the FEDER Funds (BES-2016-078290, PRX19/00289, RTI2018-094279-B-100) and in part by the European Commission (COMMANDIA SOE2/P1/F0638), action supported by Interreg-V Sudoe

Non-Matrix Tactile Sensors: How Can Be Exploited Their Local Connectivity For Predicting Grasp Stability?

Tactile sensors supply useful information during the interaction with an object that can be used for assessing the stability of a grasp. Most of the previous works on this topic processed tactile readings as signals by calculating hand-picked features. Some of them have processed these readings as images calculating characteristics on matrix-like sensors. In this work, we explore how non-matrix sensors (sensors with taxels not arranged exactly in a matrix) can be processed as tactile images as well. In addition, we prove that they can be used for predicting grasp stability by training a Convolutional Neural Network (CNN) with them. We captured over 2500 real three-fingered grasps on 41 everyday objects to train a CNN that exploited the local connectivity inherent on the non-matrix tactile sensors, achieving 94.2% F1-score on predicting stability

Tactile-Driven Grasp Stability and Slip Prediction

One of the challenges in robotic grasping tasks is the problem of detecting whether a grip is stable or not. The lack of stability during a manipulation operation usually causes the slippage of the grasped object due to poor contact forces. Frequently, an unstable grip can be caused by an inadequate pose of the robotic hand or by insufficient contact pressure, or both. The use of tactile data is essential to check such conditions and, therefore, predict the stability of a grasp. In this work, we present and compare different methodologies based on deep learning in order to represent and process tactile data for both stability and slip prediction.Work funded by the Spanish Ministries of Economy, Industry and Competitiveness and Science, Innovation and Universities through the grant BES-2016-078290 and the project RTI2018-094279-B-100, respectively, as well as the European Commission and FEDER funds through the COMMANDIA project (SOE2/P1/F0638), action supported by Interreg-V Sudoe

Predicción de la Estabilidad en Tareas de Agarre Robótico con Información Táctil

En tareas de manipulación robótica es de especial interés detectar si un agarre es estable o por el contrario, el objeto agarrado se desliza entre los dedos debido a un contacto inadecuado. Con frecuencia, la inestabilidad en el agarre puede ser como consecuencia de una mala pose de la mano o pinza robótica durante su ejecución y o una presión de contacto insuficiente a la hora de ejercer la tarea. El empleo de información táctil y la representación de ésta es vital para llevar a cabo la predicción de estabilidad en el agarre. En este trabajo, se presentan y comparan distintas metodologías para representar la información táctil, así como los métodos de aprendizaje más adecuados en función de la representación táctil escogida.Este trabajo ha sido financiado con Fondos Europeos de Desarrollo Regional (FEDER), Ministerio de Economía, Industria y Competitividad a través del proyecto RTI2018-094279-B-100 y la ayuda predoctoral BES-2016-078290, y también gracias al apoyo de la Comisión Europea y del programa Interreg V. Sudoe a través del proyecto SOE2/P1/F0638

Learning Spatio Temporal Tactile Features with a ConvLSTM for the Direction Of Slip Detection

Robotic manipulators have to constantly deal with the complex task of detecting whether a grasp is stable or, in contrast, whether the grasped object is slipping. Recognising the type of slippage—translational, rotational—and its direction is more challenging than detecting only stability, but is simultaneously of greater use as regards correcting the aforementioned grasping issues. In this work, we propose a learning methodology for detecting the direction of a slip (seven categories) using spatio-temporal tactile features learnt from one tactile sensor. Tactile readings are, therefore, pre-processed and fed to a ConvLSTM that learns to detect these directions with just 50 ms of data. We have extensively evaluated the performance of the system and have achieved relatively high results at the detection of the direction of slip on unseen objects with familiar properties (82.56% accuracy).Work funded by the Spanish Ministry of Economy, Industry and Competitiveness, through the project DPI2015-68087-R (predoctoral grant BES-2016-078290) as well as the European Commission and FEDER funds through the COMMANDIA (SOE2/P1/F0638) action supported by Interreg-V Sudoe

Using Geometry to Detect Grasping Points on 3D Unknown Point Cloud

In this paper, we focus on the task of computing a pair of points for grasping unknown objects, given a single point cloud scene with a partial view of them. The main goal is to estimate the best pair of 3D-located points so that a gripper can perform a stable grasp over the objects in the scene with no prior knowledge of their shape. We propose a geometrical approach to find those contact points by placing them near a perpendicular cutting plane to the object’s main axis and through its centroid. During the experimentation we have found that this solution is fast enough and gives sufficiently stable grasps for being used on a real service robot.This work was funded by the Spanish Government Ministry of Economy, Industry and Competitiveness through the project DPI2015-68087-R and the predoctoral grant BES-2016-078290

Fast geometry-based computation of grasping points on three-dimensional point clouds

Industrial and service robots deal with the complex task of grasping objects that have different shapes and which are seen from diverse points of view. In order to autonomously perform grasps, the robot must calculate where to place its robotic hand to ensure that the grasp is stable. We propose a method to find the best pair of grasping points given a three-dimensional point cloud with the partial view of an unknown object. We use a set of straightforward geometric rules to explore the cloud and propose grasping points on the surface of the object. We then adapt the pair of contacts to a multi-fingered hand used in experimentation. We prove that, after performing 500 grasps of different objects, our approach is fast, taking an average of 17.5 ms to propose contacts, while attaining a grasp success rate of 85.5%. Moreover, the method is sufficiently flexible and stable to work with objects in changing environments, such as those confronted by industrial or service robots.This work was funded by the Spanish Ministry of Economy, Industry and Competitiveness through the project DPI2015-68087-R (pre-doctoral grant BES-2016-078290) as well as the European Commission and FEDER funds through the COMMANDIA project (SOE2/P1/F0638), action supported by Interreg-V Sudoe